Use of metrics to control throttling and swapping in a message processing

ABSTRACT

A system and method of using metrics to control throttling and swapping in a message processing system is provided. A workload status of a message processing system is determined, and the system polls for a new message according to the workload status. The message processing system identifies a blocked instance and calculates an expected idle time for the blocked instance. The system dehydrates the blocked instance if the expected idle time exceeds a predetermined threshold.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/714,157 filed Nov. 14, 2003, entitled “SYSTEMS AND METHODS FOR USINGMETRICS TO CONTROL THROTTLING AND SWAPPING IN A MESSAGE PROCESSINGSYSTEM”, which is hereby incorporated by reference, in its entirety.

TECHNICAL FIELD

The present invention relates generally to software development. Moreparticularly, the present invention relates to a method and system foroptimizing system performance for a given workload by controlling thearrival of new work and managing the processing of existing work. Evenmore particularly, the present invention relates to a method and systemfor using performance metrics for controlling throttling of new messagesand swapping of existing processes in a message processing system.

BACKGROUND

An entity may use a software application, such as a web service, toautomate various processes and to interact with other entities in adistributed environment, such as the Internet or World Wide Web, or acorporate intranet or wide area or local area network. To ensure thatsuch interactions are accomplished successfully, one or more protocolsshould be in place for carrying messages to and from participants, andspecific applications should also be in place at each participant's end.Such interactions are message-driven. For example, a buyer sends apurchase order to a seller. The seller then checks its inventory todetermine if it can provide the ordered items. If so, the seller sendsan acknowledgement back to the buyer with a price. Finally, the buyeraccepts or rejects the seller's offer (and/or possibly places anotherorder). As evident in this example, each participant's applicationreacts to the receipt of messages.

Many such interactions and processes may take place at a given time. Inaddition, the same interaction may be carried out at the same timebetween different parties. For example, the buyer may send severalpurchase orders to different sellers. Thus, at any given time multipleinstances of each interaction and/or an instance of many differentprocesses may need to be processed. A conventional system for processingsuch messages may become overburdened when messages arrive too quicklyfor the system to handle, or when too many interactions or otherprocesses are occupying resources without making progress towardcompletion.

An example of a situation where a message processing system may becomeoverburdened is when the arrival rate of messages becomes too great. Insuch a situation, the system expends a large percentage of availableresources on servicing the arrival of the messages —such as determiningto which instance each message belongs, whether the message requires thecreation of a new instance, and so forth—which leaves few resourcesavailable to actually process the messages. A system in such a state issaid to be “thrashing.” In addition, a large amount of processing poweris typically required to process a message, which complicates the issueof handling a large number of arriving messages. For example, whenprocessing a message the system may need to update one or more databaseentries based on the message, transmit one or more messages, and soforth. Therefore, as the processing power required to process a messageincreases, the number of incoming messages required to overburden themessage processing system decreases. When a system receives a greaternumber of messages than it can adequately process, the system canproduce errors, loose data, or may simply become very slow and/orunresponsive. For example, in the above buyer/seller scenario, thetransaction may be delayed, the messages with the buyer's and/orseller's instructions may be lost or corrupted, and/or the entiretransaction aborted as a result of the system's inability to adequatelyprocess the messages.

A conventional message processing system that experiences suchoverburdening issues is also especially vulnerable to “denial ofservice” computer attacks. In such attacks, a hostile party transmits avery large number of messages to a particular message processing system.The system becomes overwhelmed when attempting to process the messages,and as a result the process(es) the system is supposed to be running areadversely affected or interrupted entirely.

Accordingly, and in light of the above shortcomings, what is needed is asystem and method for throttling, or controlling the arrival rate of newmessages, and selectively moving work that is already being processedinto secondary storage. More particularly, what is needed is an agentthat retrieves a new message and directs such a message to its instance.Even more particularly, what is needed is a controller that usesfeedback from the instance state to control instance swapping tooptimize performance of the message processing system

SUMMARY

In light of the foregoing limitations and drawbacks, a system and methodof using metrics to control throttling and swapping in a messageprocessing system is provided. In the inventive method, a workloadstatus of a message processing system is determined, and the systempolls for a new message according to the workload status. The messageprocessing system identifies a blocked instance and calculates anexpected idle time for the blocked instance. The system dehydrates theblocked instance if the expected idle time exceeds a predeterminedthreshold. In one embodiment, the workload status of the system isupdated according to the dehydration. In another embodiment, the systempolls for a new message at a frequency that is inversely proportional tothe system's workload. In yet another embodiment, the system polls onlyfor a new non-activation message.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofpreferred embodiments, is better understood when read in conjunctionwith the appended drawings. For the purpose of illustrating theinvention, there is shown in the drawings exemplary embodiments of theinvention; however, the invention is not limited to the specific methodsand instrumentalities disclosed. In the drawings:

FIG. 1 is a diagram illustrating an exemplary computing environment inwhich aspects of the invention may be implemented;

FIG. 2 is a diagram illustrating an exemplary computing network in whichaspects of the invention may be implemented;

FIG. 3 is a graph illustrating resource management in a messageprocessing system in accordance with one embodiment of the presentinvention;

FIG. 4 is a block diagram illustrating a configuration of exemplarysoftware components in which aspects of the invention may beimplemented; and

FIGS. 5A-B are flowcharts illustrating an exemplary method of managingmessage processing in accordance with one embodiment of the presentinvention detailed description of illustrative embodiments

OVERVIEW

A system and method of improving the efficiency of a message processingsystem is disclosed herein. Performance data, or metrics, are used todetermine the overall workload of such a system and to determine whetherthe system is overburdened with messages. In response to such adetermination, the number and/or type of new messages may be restrictedso as to enable the system to process the existing work. In addition,metrics are used to predict how long an idle process will remain idle,thereby enabling a determination as to whether the system should movethe process out of active memory and into secondary storage, therebyimproving system performance. Such determinations can be made in astatic or dynamic manner.

Before discussing the invention in detail, we will first describeexemplary computing and network environments in which the invention maybe advantageously practiced.

Exemplary Computing Environment

FIG. 1 illustrates an example of a suitable computing system environment100 in which the invention may be implemented. The computing systemenvironment 100 is only one example of a suitable computing environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should the computing environment100 be interpreted as having any dependency or requirement relating toany one or combination of components illustrated in the exemplaryoperating environment 100.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network or other data transmission medium. In adistributed computing environment, program modules and other data may belocated in both local and remote computer storage media including memorystorage devices.

With reference to FIG. 1, an exemplary system for implementing theinvention includes a general purpose computing device in the form of acomputer 110. Components of computer 110 may include, but are notlimited to, a processing unit 120, a system memory 130, and a system bus121 that couples various system components including the system memoryto the processing unit 120. The system bus 121 may be any of severaltypes of bus structures including a memory bus or memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures. By way of example, and not limitation, such architecturesinclude Industry Standard Architecture (ISA) bus, Micro ChannelArchitecture (MCA) bus, Enhanced ISA (EISA) bus, Video ElectronicsStandards Association (VESA) local bus, and Peripheral ComponentInterconnect (PCI) bus (also known as Mezzanine bus).

Computer 110 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 110 and includes both volatile and non-volatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes both volatileand non-volatile, removable and non-removable media implemented in anymethod or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media includes, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can accessed by computer 110. Communication media typicallyembodies computer readable instructions, data structures, programmodules or other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia. The term “modulated data signal” means a signal that has one ormore of its characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared and other wireless media. Combinations of any of the aboveshould also be included within the scope of computer readable media.

The system memory 130 includes computer storage media in the form ofvolatile and/or non-volatile memory such as ROM 131 and RAM 132. A basicinput/output system 133 (BIOS), containing the basic routines that helpto transfer information between elements within computer 110, such asduring start-up, is typically stored in ROM 131. RAM 132 typicallycontains data and/or program modules that are immediately accessible toand/or presently being operated on by processing unit 120. By way ofexample, and not limitation, FIG. 1 illustrates operating system 134,application programs 135, other program modules 136, and program data137.

The computer 110 may also include other removable/non-removable,volatile/non-volatile computer storage media. By way of example only,FIG. 1 illustrates a hard disk drive 140 that reads from or writes tonon-removable, non-volatile magnetic media, a magnetic disk drive 151that reads from or writes to a removable, non-volatile magnetic disk152, and an optical disk drive 155 that reads from or writes to aremovable, non-volatile optical disk 156, such as a CD-ROM or otheroptical media. Other removable/non-removable, volatile/non-volatilecomputer storage media that can be used in the exemplary operatingenvironment include, but are not limited to, magnetic tape cassettes,flash memory cards, digital versatile disks, digital video tape, solidstate RAM, solid state ROM, and the like. The hard disk drive 141 istypically connected to the system bus 121 through a non-removable memoryinterface such as interface 140, and magnetic disk drive 151 and opticaldisk drive 155 are typically connected to the system bus 121 by aremovable memory interface, such as interface 150.

The drives and their associated computer storage media, discussed aboveand illustrated in FIG. 1, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 110. In FIG. 1, for example, hard disk drive 141 is illustratedas storing operating system 144, application programs 145, other programmodules 146, and program data 147. Note that these components can eitherbe the same as or different from operating system 134, applicationprograms 135, other program modules 136, and program data 137. Operatingsystem 144, application programs 145, other program modules 146, andprogram data 147 are given different numbers here to illustrate that, ata minimum, they are different copies. A user may enter commands andinformation into the computer 110 through input devices such as akeyboard 162 and pointing device 161, commonly referred to as a mouse,trackball or touch pad. Other input devices (not shown) may include amicrophone, joystick, game pad, satellite dish, scanner, or the like.These and other input devices are often connected to the processing unit120 through a user input interface 160 that is coupled to the systembus, but may be connected by other interface and bus structures, such asa parallel port, game port or a universal serial bus (USB). A monitor191 or other type of display device is also connected to the system bus121 via an interface, such as a video interface 190, and video interface190 and monitor 191 may be associated with a graphics interface 182, agraphics processing unit (GPU) 184, and video memory 186 in a generallyknown manner. In addition to the monitor, computers may also includeother peripheral output devices such as speakers 197 and printer 196,which may be connected through an output peripheral interface 195.

The computer 110 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer180. The remote computer 180 may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 110, although only a memory storage device 181 has beenillustrated in FIG. 1. The logical connections depicted include a localarea network (LAN) 171 and a wide area network (WAN) 173, but may alsoinclude other networks. Such networking environments are commonplace inoffices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN networking environment, the computer 110 is connectedto the LAN 171 through a network interface or adapter 170. When used ina WAN networking environment, the computer 110 typically includes amodem 172 or other means for establishing communications over the WAN173, such as the Internet. The modem 172, which may be internal orexternal, may be connected to the system bus 121 via the user inputinterface 160, or other appropriate mechanism. In a networkedenvironment, program modules depicted relative to the computer 110, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 1 illustrates remoteapplication programs 185 as residing on memory device 181. It will beappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

Exemplary Distributed Computing Frameworks Or Architectures

Various distributed computing frameworks have been and are beingdeveloped in light of the convergence of personal computing and theInternet. Individuals and business users alike are provided with aseamlessly interoperable and web-enabled interface for applications andcomputing devices, making computing activities increasingly web browseror network-oriented.

For example, MICROSOFT®'s .NET platform includes servers, building-blockservices, such as web-based data storage, and downloadable devicesoftware. Generally speaking, the .NET platform provides (1) the abilityto make the entire range of computing devices work together and to haveuser information automatically updated and synchronized on all of them,(2) increased interactive capability for web sites, enabled by greateruse of XML rather than HTML, (3) online services that feature customizedaccess and delivery of products and services to the user from a centralstarting point for the management of various applications, such ase-mail, for example, or software, such as Office .NET, (4) centralizeddata storage, which will increase efficiency and ease of access toinformation, as well as synchronization of information among users anddevices, (5) the ability to integrate various communications media, suchas e-mail, faxes, and telephones, (6) for developers, the ability tocreate reusable modules, thereby increasing productivity and reducingthe number of programming errors, and (7) many other cross-platformintegration features as well.

While exemplary embodiments herein are described in connection withsoftware residing on a computing device, one or more portions of theinvention may also be implemented via an operating system, API, ormiddleware software between a coprocessor and requesting object, suchthat services may be performed by, supported in, or accessed via all of.NET's languages and services, and in other distributed computingframeworks as well.

Introduction to Web Services

As noted above, entities—such as businesses—are more frequentlyinteracting via distributed environments, such as the Internet or WorldWide Web. For example, a consumer may want to know the prices of rentalcars for an upcoming trip. The consumer may request the prices throughan intermediate business on the Internet, such as a travel website. Theintermediate business, upon receiving the consumer's request, sendsprice quote requests to a variety of rental car businesses. Afterresponses are received from the rental car businesses, the intermediatebusiness then sends the responses to the consumer. The consumer may thenreserve a car and pay for the reservation by way of the intermediatebusiness. As noted above, such business processes are implemented usingmessages. For example, the consumer's request to the intermediatebusiness is implemented as an electronic message to the intermediatebusiness that contains the particulars of the proposed car rental, forexample: rental dates and times, type of car, additional insurancedesired, consumer's name and address, credit card information and/or thelike. The intermediate business' price quote request is another message,as are the responses from the rental car businesses—both to theintermediate business from the rental car businesses and from theintermediate business to the consumer—and the reservation ultimatelysent by the consumer.

Distributed computer languages are designed to automate businessprocesses and the messages used to carry out such processes. One suchlanguage is XLANG/s, which is described in more detail below. As may beappreciated, it is important that the business processes that areimplemented using XLANG/s execute properly, and as efficiently aspossible. A system processing messages in accordance with a languagesuch as XLANG/s should be capable of managing its processing of arrivingmessages in such a way as to be able to continue processing on existingmessages. Providing a method and system for managing such messageprocessing in an efficient manner is an application of an embodiment ofthe present invention.

XLANG/s Introduction

An embodiment of the present invention is implemented in a distributedcomputing language. As noted above, one such language that is compatiblewith an embodiment of the present invention is XLANG/s. XLANG/s is alanguage that describes the logical sequencing of business processes, aswell as the implementation of the business process by using varioustechnology components or services. XLANG/s is described in more detailthan is disclosed herein in a document titled “XLANG/s LanguageSpecification,” Version 0.55, Copyright© Microsoft 1999-2000, and adocument titled “XLANG Web Services For Business Process Design,” SatishThatte, Copyright© Microsoft Corporation 2001, both of which are herebyincorporated by reference in their entirety. The XLANG language isexpressed in Extensible Markup Language (XML). XLANG/s is a modern,domain specific, special purpose language used to describe businessprocesses and protocols. XLANG/s is also a declarative language, whichmeans that it defines an explicit instruction set that describes andimplements steps in a business process, the relationship between thosesteps, as well as their semantics and interactions. In addition, XLANG/scode is not just descriptive; it is also designed to be executable.Because of the declarative nature of XLANG/s and its specific semantics,the resulting executable code is deterministic; that is, the behavior ofthe running business process is well defined by the semantics of thecollection of XLANG/s instructions. Therefore, by examining XLANG/s codeone is able to determine the business process that is carried out bysuch code. As noted above, the definition of such a business process inexecutable form is an “orchestration service.”

XLANG/s is compatible with many Internet standards. XLANG/s is designedto use XML, XSLT, XPATH, XSD (XML Schema Definition) and WSDL (WebServices Description Language) as supported standards and has embeddedsupport for working with .NET based objects and messages. WSDL isdescribed in a document titled “Web Services Description Language (WSDL)1.1,” W3C Note January 2001, by Microsoft and IBM Research, Copyright©2000 Ariba, International Business Machines Corporation, Microsoft, andis hereby incorporated by reference in its entirety. The XLANG/slanguage is syntactically similar to C#, thus a C# specification mayalso be referenced as an aid to understanding the exact syntax. Thesemantics embodied in XLANG/s are a reflection of those defined in adocument entitled “Business Process Execution Language for WebServices,” Version 1.1, dated Mar. 31, 2003, published by Microsoft, IBMand BEA for the definition of Business Process semantics, which is alsohereby incorporated by reference in its entirety. The Business ProcessExecution Language for Web Services specification is commonly referredto as the “BPEL4WS” specification. As may be appreciated, therefore, theuse of XLANG/s is most advantageous when applied to a business process.

XLANG/s defines a rich set of high-level constructs used to define abusiness process. XLANG/s statements generally fall into one of twocategories: simple statements that act on their own, such as receive orsend, and complex statements that contain or group simple statementsand/or other complex statements. XLANG/s also supports low-level datatypes such as strings or integers, for example. High-level data typesare also defined such as, for example, messages, ports (locations towhich messages are sent and received), correlations and service links.The data types are used to rigorously define the semantics associatedwith the business process.

As noted above, a XLANG/s service communicates with the outside world bysending and/or receiving messages. The message type is the structuraldefinition of such a message. Messages are acted upon by operations(e.g., receive, response), and an operation may be either a singleasynchronous message or a request-response pair of messages. Operationsmay be either incoming or outgoing. For example, a seller may offer aservice/product that begins an interaction by accepting a purchase order(from a potential buyer) by way of an input message. The seller may thenreturn an acknowledgement to the buyer if the order can be fulfilled.The seller may send additional messages to the buyer (e.g., shippingnotices, invoices). Typically, these input and output operations occurin accordance with a defined sequence, referred to as a “serviceprocess.” The seller's service remembers the state of each purchaseorder interaction separately from other similar interactions. This isparticularly advantageous in situations in which the buyer may beconducting many simultaneous purchase processes with the same seller.Also, each instance of a service process may perform activities in thebackground (e.g., update inventory, update account balance) without thestimulus of an input operation.

A service process may represent an interaction utilizing severaloperations. As such the interaction has a well-defined beginning andend. This interaction is referred to as an instance of the service. Aninstance can be started in either of two ways. A service can beexplicitly instantiated using some implementation-specific functionalityor a service can be implicitly instantiated with an operation in itsbehavior that is meant to be an instantiation operation. A serviceinstance terminates when the process that defines its behaviorterminates.

Services are instantiated to act in accordance with a history of anextended interaction. Messages sent to such services are delivered notonly to the correct destination port, but to the correct instance of theservice that defines the port. A port is an end point where messages aresent and received by a service. The infrastructure hosting the servicesupports this routing, thus avoiding burdening every serviceimplementation with the need to implement a custom mechanism forinstance routing.

Turning now to FIG. 2, a simplified, exemplary computer network forenabling communications between two entities is illustrated. A firstcomputer 220, which is any type of computing device such as, forexample, computer 110 as disclosed above in connection with FIG. 1, aspecial-purpose computer or the like, is operatively connected to anetwork 210 by way of communications link 222. First computer 220 isalso operatively connected to a database 228, which in one embodimentcontains process-specific information as will be discussed below.Database 228 may be internal to the first computer 220, or may belocated on another device. Network 210 may be any type of network forinterconnecting a plurality of computing devices, and may be anintranet, the Internet, etc. Communications link 222 may comprise anytype of communications medium, whether wired, wireless, optical or thelike. Second computer 230, like first computer 220, may be any type ofcomputing device, and is operatively connected to network 210 by way ofcommunications link 232. As can be seen in FIG. 2, second computer 230is also operatively connected to a database 234.

Communications link 232, like communications link 222, may be any typeof communications medium. In one embodiment, communications links 222and 232 are the same type of communications medium, while in anotherembodiment the medium employed by each communications link 222 and 232is different. In FIG. 2, it can be seen that first computer 220 is alsooperatively connected to computers 224 and 226 by way of database 228.As may be appreciated, additional computers may be operatively connectedto second server 230 as well (not shown in FIG. 2 for clarity). It willbe appreciated that, although described herein as computers 220, 224,226 and 230, such computers may be a client or a server computer, or acombination of both, depending on the exact implementation of thecomputer network and the relationship between computers during atransaction. It will also be appreciated that any combination orconfiguration of computers and databases is equally consistent with anembodiment of the present invention.

For example, consider a typical supply chain situation in which a buyersends a purchase order to a seller. The buyer sends the message from,for example, first computer 220 to the seller's second computer 230 byway of the network 210 and communications links 222 and 232. Assume, forexample, that the buyer and seller have a stable business relationshipand are statically configured—by way of settings stored in databases 228and 234—to send documents related to the purchasing interaction to theURLs associated with the relevant ports. When the seller returns anacknowledgement for the order, the acknowledgement is routed to thecorrect service instance at the buyer's end at first computer 220 or,optionally, another computer such as additional computer 224 or 226, byway of database 228. One way to implement such a routing is to carry anembedded token (e.g., cookie) in the order message that is copied intothe acknowledgement for correlation. The token may be in the message“envelope” in a header or in the business document (purchase order)itself. The structure and position of the tokens in each message can beexpressed declaratively in the service description. This declarativeinformation allows a XLANG/s-compliant infrastructure to use tokens toprovide instance routing automatically.

In one embodiment of the present invention, another function ofdatabases 228 and 234 is to serve as a repository for persisted stateinformation for any instances of an orchestration service. For example,first computer 220 transmits a message to second computer 230 inaccordance with a service process for a currently-running orchestrationservice. Upon first computer 220 sending the message, database 228records the state information for the instance. In such a case, thestate information may record that a message has been sent to secondcomputer 230, the content of a message, and that the first computer 220is waiting for a response. Upon the occurrence of second computer 230receiving first computer's 220 message, database 234 records the stateinformation for the instance. In the present example, such stateinformation indicates that a message from first computer 220 wasreceived, the contents of such message and that a response must begenerated.

Accordingly, if a communication error or power interruption occurs, uponresolution of the problem first computer 220 will know that it has senta message to second computer 230 and is currently waiting for aresponse, and second computer 230 will know that it has received amessage from first computer 220 and must generate a response. Inaddition, the storage of state information for an instance of anorchestration service enables the processing of long-runningtransactions. For example, and as noted above, the business processbeing implemented by the orchestration service may take a long period oftime to generate a response to the message. In such a case, both firstand second computers 220 and 230 can process other orchestrationservices and then return to the instance at the correct point in itsservice process once the response is generated.

During its lifetime, a service instance may typically hold one or moreconversations with other service instances representing otherparticipants involved in the interaction. Conversations may use asophisticated transport infrastructure that correlates the messagesinvolved in a conversation and routes them to the correct serviceinstance. In many cases, correlated conversations may involve more thantwo parties or may use lightweight transport infrastructure withcorrelation tokens embedded directly in the business documents beingexchanged. XLANG/s addresses correlation scenarios by providing a verygeneral mechanism to specify correlated groups of operations within aservice instance. A set of correlation tokens can be defined as a set ofproperties shared by all messages in the correlated group. Such a set ofproperties is called a correlation set.

DESCRIPTION OF EMBODIMENTS OF THE PRESENT INVENTION

The description contained herein is not intended to limit the scope ofthis patent. Rather, the inventors have contemplated that the claimedsubject matter might also be embodied in other ways, to includedifferent elements or combinations of elements similar to the onesdescribed in this document, in conjunction with present or futuretechnologies.

Accordingly, it will be appreciated that an embodiment of the presentinvention is equally compatible with any type of computer programminglanguage that is capable of automating a distributed process such as,for example, a distributed business process. Therefore, the descriptionherein reflecting the use of XLANG/s is merely illustrative, as any typeof equivalent language is consistent with an embodiment of the presentinvention. In addition, it will be appreciated that the use herein ofXLANG/s-specific terminology is done for the sake of clarity and in noway implies that only XLANG/s components or functionality may be used inconnection with the present invention. Accordingly, any type of similarcomponents and/or functionality may be used in accordance with anembodiment of the present invention.

In the discussion to follow, it will be appreciated that detailspertaining to the automation of distributed processes such as, forexample, the programming, configuration and implementation of suchprocesses in software should be known to one of skill in the art and aretherefore omitted herein for clarity.

As noted above, a message processing system such as, for example,software that is automating a distributed business process, can receivea greater number of messages than it is capable of processing at thesame time. In such a situation, the system devotes too great apercentage of resources toward handling the arrival of the new messagesinstead of processing the messages that are already active. As a result,thrashing and/or blocking may occur. As noted above, thrashing refers toa state where a system is overwhelmed and spends most of its time movingdata into and out of memory rather than performing any usefulcomputations. Blocking refers to a process that remains idle whilewaiting for something to occur such as, for example, the receipt of amessage.

Referring now to FIG. 3, a graph 300 illustrating resource management ina message processing system in accordance with one embodiment of thepresent invention is provided. In FIG. 3, X-axis 302 corresponds to ametric used to measure the workload of a message processing system suchas, for example, memory occupied by operations, processor power in useand the like. X-axis 302 varies from a value of zero to a maximum value,denoted by “Max.” “M” corresponds to a target utilization of theavailable memory and resources of the system. Y-axis 304 corresponds tothe deviation between a given workload and the target utilization M.Accordingly, the value of the deviation is zero at the targetutilization M.

It will be appreciated that in FIG. 3 the region 310 thereforecorresponds to an underutilization of system resources, while the region312 corresponds to an overutilization of system resources. Operation ofsuch a system within the region 312 therefore carries with it the riskof the aforementioned thrashing and blocking. Advantageously, oneembodiment of the present invention provides the ability to operate amessage processing system as close to the target utilization M aspossible. In addition, and as will be discussed below in connection withFIGS. 5A-B, one embodiment of the present invention dynamicallydetermines the target utilization M based on system resources andhistorical performance data. Accordingly, an embodiment of the presentinvention determines the target utilization M, and then performs“throttling” and “swapping” as necessary to drive the performance of thesystem toward the target utilization. Throttling refers to the system'scontrol of the arrival rate of new messages or processes to be executed,and swapping refers to moving—or dehydrating—instances that are alreadybeing processed (but are currently idle) to secondary storage to free upsystem resources.

It will be appreciated that dehydrating an instance carries with itcertain processing “overhead” that should be accounted for to ensureefficient operation of the system. For example, in one embodimentdehydration involves storing all of the instance states to a databaseand clearing active memory of instance-related data. Such tasks takeprocessing time to perform, and therefore in one embodiment of thepresent invention such dehydration is only to occur when it isdetermined that the processing and time savings to be achieved by such adehydration is greater than the associated processing overheadcorresponding to performing the dehydration. Otherwise, certaininstances will end up taking longer to process because the acts of de-and rehydrating the instance take more time and processing power thanwould have been consumed if the instance had simply remained in activememory. It will also be appreciated that such a determination may beweighted by certain performance data, or metrics, that can serve aspredictors of time and/or processing savings that may result from such adehydration.

A message processing system such as, for example, XLANG/s-based businessautomation software, has a structure that may be leveraged by anembodiment of the present invention to ascertain useful metrics for theabove determination of the target utilization M. For example, a systemin which aspects of one embodiment of the present invention may beimplemented can concurrently execute many instances. The arrival of amessage may create a new instance or may be directed to an existinginstance, and each instance may process many different messages over theinstance's lifetime. In addition, instances are instantiations of one ofa set of services. The set of available services is fixed—or slowlyevolving—thus the same services are run repeatedly on different inputs,thereby providing a history of activity that can be used to predictbehavior of future executions. Languages such as XLANG/s have componentsthat measure and record certain metrics regarding the operation of, forexample, message processing. As will be discussed in connection withFIGS. 5A-B, such metrics are used to provide resource utilizationinformation, predicted idleness for a given instance, and the like.

Accordingly, in one embodiment of the present invention, a messagingagent is provided that “pulls” in a new message and “pushes” such amessage to an independent instance. Likewise, a dehydration controlleris also provided that monitors system resource usage and controlsinstance swapping. The agent uses feedback from the instance state tocontrol any throttling, and the dehydration controller uses the feedbackto select swapping candidates. As will be discussed below in connectionwith FIGS. 5A-B, the throttling algorithm is based on a metric for theamount of unfinished work in the system and the swapping algorithm isbased on a metric for predicting idle duration.

Referring now to FIG. 4, a block diagram illustrating a configuration ofexemplary software components in which aspects of the present inventionmay be implemented is shown. In FIG. 4, a process engine 400 that isrunning one or more instances 401 and one or more dehydration controller403 is illustrated. It will be appreciated that an engine 400 is, in oneembodiment, a service that runs XLANG/s schedule instances 401 andcontrols the activation, execution, dehydration and rehydration of suchinstances 401. The engine 400, or one or more instances 401 being run bythe engine 400, defines a send port 404 a and a receive port 404 b. Asmay be appreciated, each instance 401 may define one or more send andreceive ports 404, and so therefore the engine 400 may define aplurality of such ports 404. In addition, the engine 400 may run one ormore components that monitor and record performance data (metrics)relating to the processing of messages and other functions. As will beappreciated, in one embodiment such metrics are used to determinewhether to throttle messages or swap instances, as will be discussedbelow in connection with FIGS. 5A-B. The dehydration controller 403, asnoted above and as will also be discussed in greater detail below inconnection with FIGS. 5A-B, receives feedback from the instance 401state to select swapping candidates from all instances 401 and forwardssuch candidates to the engine 400 for dehydration.

A transport 420 is a set of services that includes network protocols andapplication integration components, which enable a server running theengine 400 to send documents to organizations or applications whether ornot the applications are capable of communicating directly with theserver by using, for example, a COM interface. In one embodiment,XLANG/s supports, for example, the file, HTTP, HTTPS, and SMTP networkprotocols, as well as Message Queuing and the like. It will beappreciated that the transport 420 transmits and receives messagesaccording to the particular instance being processed by the engine 400.As was the case with engine 400, the transport 420 defines a send port424 a and a receive port 424 b, by which the transport 420 communicateswith other software or hardware components, or other computing devices.

The messaging agent 430, in one embodiment, is a computing abstractionthat controls access to the message box database 440 and is capable ofimplementing relevant portions of the method to be discussed below inconnection with FIGS. 5A-B. It will be appreciated that the messagingagent 430, in one embodiment, performs such activities in conjunctionwith the transport 420. In an embodiment implemented in XLANG/s, forexample, the messaging agent 430 and transport 420 interact with eachother by “publishing” messages and/or other data from the transport 420to the messaging agent 430, and by “subscribing” to messages and/orother data from the messaging agent 430 to the transport 420.

The message box database 440 contains, for example, message informationpertaining to a XLANG/s instance being run by the engine 400. Themessage box database 440, or another database or memory device, is alsocapable of storing the instance when or if such instance is dehydrated.Thus, the exemplary software and hardware components introduced in thediscussion of FIG. 4 above will now be discussed in the context of themethod of FIGS. 5A-B. As noted above, in a typical process automationapplication an instance such as, for example, instance 401 as discussedabove in connection with FIG. 4, is a program executing a service. Inone embodiment, an instance is “pushed” a message. In other words, aninstance is delivered a message when the message arrives, rather thanthe instance asking for a message when the instance is ready for such amessage. The instance contains a set of one or more tasks, which may bedynamically changing. If the instance reaches a state in which itrequires input that is not currently available (e.g., the instance iswaiting to receive a message) then the task blocks. The place in code atwhich the instance blocks is the instance's blocking location.

A messaging agent such as, for example, message agent 430, supplies eachrunning instance with messages as such messages arrive by way oftransport 420 or the like. The message agent 430 retrieves each newmessage and assigns it to the proper instance. If an appropriateinstance does not exist then the message agent 430 creates a newinstance of the correct service type and delivers the message to the newinstance. A message that necessitates the creation of a new instance isreferred to as an activation message. Thus, the message agent 430controls the arrival of messages. In one embodiment, the message agent430 accomplishes this task by polling for new messages at given rate. Amessage agent 430 according to an embodiment has the ability to; forexample, control the rate at which the agent polls for new messages, andalso to selectively poll only for non-activation messages.

As will be discussed below in connection with FIGS. 5A-B, the throttlingalgorithm is executed by the message agent 430. According to oneembodiment of the present invention, the message agent 430 operates inone of three states: a state where the message agent 430 aggressivelypolls for all messages at a high frequency and in multiple threads andassigns such messages to instances (referred to hereinafter forreference as state “A”), a state where the message agent 430 polls foronly non-activation messages at a frequency adaptively determined by theavailability of such messages in the message box 440 and assigns suchmessages to instances (state “B”), and a state where the message agent430 completely suspends polling of any message from the message box 440until the overall usage of system resources returns to normal and thesystem is able to accept and process new messages (state “C”).

Turning now to FIG. 5A, a flowchart depicting a method 500 forcontrolling throttling and swapping according to an embodiment of thepresent invention is illustrated. It will be appreciated that the dottedlines surrounding steps 501-505 indicate that, while steps 501-505 arelocated at the beginning of method 500 for clarity, such steps 501-505may take place at any time during the method 500, including taking placeat the same time as steps 510-550, as well as steps 555-570 of FIG. 5B.In fact, steps 501-505 may take place completely independently from theremaining steps of the method 500. Thus, it will be appreciated that thethrottling functionality of one embodiment of the present inventionpolls for new messages, while the swapping functionality dehydratesblocked instances so as to free up system resources. In an embodiment,each action is interrelated, as the freeing up of system resourcesaffects which state (e.g., A, B or C) governs the polling for newmessages, and the arrival rate of new messages affects the overall usageof system resources. Thus, a type of feedback loop is created, wherebyan embodiment of the present invention takes actions based on changingmetrics to keep the system operating at or near an optimum workload.

Therefore, at step 501, the method 500 determines a state that indicatesthe polling rate at which new messages should be retrieved. In oneembodiment, the message agent 430 moves from state to state based on acurrent value of a stress metric, J, and two predefined values: alow-watermark, L, and a high-watermark, H. The message agent 430 beginsin, for example, state A. In state A or B, if J becomes greater than Hthen the agent moves to state C. When the message agent 430 is operatingin state C, if J drops below H then the agent moves to state B. When themessage agent 430 is operating in state B, if J drops below L then theagent moves to state A.

As noted above, when the message agent 430 is in state B or C, no newinstances are created; however existing instances can continue to makeprogress. In one embodiment, the purpose of J is, for example, to be ameasure of the amount work remaining to be completed by the instances.Any type of work measurement may be used while remaining consistent withan embodiment of the present invention. For example, J may beimplemented as the number of unprocessed messages, or the number ofexecuting tasks. It will be appreciated that the values for L and H, aswell as the high and low frequency polling rates are constant parametersdetermined by the choice of metric and based on the size andcapabilities of the hosting environment. Thus, such values and rates maybe any rates that may be assigned, based on the software and hardwareconfiguration that is implementing the method 500.

Accordingly, at step 503 the method 500 polls for messages according tothe determination of step 501. It will be appreciated that during such apoll, the method 500 may receive any number of messages, including noneat all. At step 505, the method receives a message. It will beappreciated that such a message may be an initiation message if suchmessage does not correspond to a currently-running instance. Therefore,step 505 may include additional steps necessary to create an instanceand/or to route the message to a currently-running instance.

At step 510, a determination is made as to whether an instance inquestion is blocked. The determination of step 510 may be made either asa result of receiving a message, or may be performed on a periodic orother basis according to the dehydration controller's 403 functionality.As discussed above, an instance may be blocked as a result of, forexample, waiting to receive a message. Thus, it will be appreciated thatin most embodiments, the blocked condition of an instance is not theresult of receiving a message at step 505. If the instance is notblocked, then the method 500 processes the message (if a message wasreceived) at step 535 according to the message system's usual protocols.Once the message has been processed at step 535, or if no message wasreceived, the method 500 proceeds to step 545, which will be discussedin greater detail below.

At step 515, a determination is made as to whether any segments of theinstance are executable, or if all of such segments are blocked andtherefore have no processing activity. If such instance is completelyblocked, the method 500 continues to step 520. If any segments areexecutable, the method 500 proceeds to step 545, below. It will beappreciated that in one embodiment only completely blocked instances canbe candidates for swapping, as otherwise the previously-discussedoverhead associated with swapping offsets any resource savings that mayoccur as a result of such swapping. However, application specifics maypermit, in some embodiments, only partially blocked instances to beswapped.

At step 520, an idleness prediction is generated. As discussed above,such a prediction permits the method 500 to make a determinationwhether, on average, the processing and resource savings that willresult from swapping a blocked instance will be worth the processingoverhead associated with such swapping. As also discussed above, thedehydration controller 403 determines when an instance needs to beswapped and which instance, out of all instances, should be swapped. Thedehydration controller 403 decides to swap based on its monitoring ofthe current resource usage of the system, and determines which instanceto swap based on a prediction of how long the instance is likely toremain idle. As noted above, in one embodiment metrics regarding, forexample, the performance of the system are used to make suchdeterminations. Such metrics may include, for example: memory usage,processor resources used, historical data regarding blocking time andthe like, and so forth. In addition, such metrics may be recorded withrespect to the entire system, a particular process or processes, aservice, and instance, and/or the like. For example, in one embodimentthe system records the idle time spent at each blocking location for aparticular service. The service then uses such information whendetermining the expected idle time at a blocking location during aparticular instance of the service.

As also noted above, a service S may have one or more instances, I(S),running at any given time. If such an instance I(S) is idle then theinstance I(S) contains a set of blocked tasks that are waiting at a setof blocking locations, B_(I(S)). The expected idle time for I(S),denoted t_(I(S)), can therefore be computed based on the past behaviorof instance I(S) when the instance's I(S) tasks were similarly blocked.One such function for computing t_(I(S)) is simply to treat thelocations independently; that is, to take the minimum of the averagepast idle time at the blocking location B_(I(S)) minus the time thatI(S) has already been blocked at the location B_(I(S)).

At step 525, the dehydration controller 403 determines whether theexpected idle time t_(I(S)) exceeds a threshold T, which may bepredetermined and/or determined as needed in a dynamic manner. Thedehydration controller 403 maintains the threshold value T such that ift_(I(S))>T then I(S) is swappable. As may be appreciated, any suchcomputation for determining t_(I(S)), T and the like so as to determinea value against which the idle nature of an instance I(S) may be usedwhile remaining consistent with an embodiment of the present invention.

In an embodiment, the dehydration controller 403 re-computes T at fixedintervals, on demand, or the like. If T≧maxT then no swapping occurs. IfT≦minT then all idle instances are swapped out. Otherwise T is adjustedup or down according to how far from ideally the system is using itsresources. The value of T can be computed based on any metric formeasuring system resource consumption. For example, one embodiment thatemploys memory usage as a metric for such computations occurs asfollows. First, the current memory consumption level, m is computed.Then a determination is made as to whether m<minM and, if so, T is setto maxT. If m>maxM then T is set to 0, and if m<M then T=T+ΔT. Finally,if m>M then T=T−ΔT. It will be appreciated that minM<M<maxM are constantvalues for the minimum, target (ideal value), and maximum memoryconsumption levels, respectively, and ΔT is a function of m, T, minM, Mand maxM. It will also be appreciated that, in one embodiment, ΔT hasthe properties of being negative when m>M, positive when m<M andconverging monotonically toward a zero value when m approaches M.

Thus, if the determination of step 525 is that t_(I(S))>T, the instanceI(S) is dehydrated at step 530. It will be appreciated that any mannerof instance dehydration is consistent with an embodiment of the presentinvention. As noted above, in one embodiment, dehydration consists ofsaving data related to the instance I(S), including all stateinformation, to a database or other stable memory device, and removingall or part of such data from active memory. In addition, suchdehydration may take place immediately, or the instance may be placed ina dehydration queue, whereby the instance is dehydrated when sufficientsystem resources are available for such dehydration. After the instanceI(S) has been dehydrated, the method 500 proceeds to step 545. At step545, and in one embodiment, normal processing and/or polling continuesuntil the method 500 needs to be resumed. In addition, at step 545, alldehydrated instances I(S) are monitored to determine whether a newmessage for a dehydrated instance I(S) has arrived. If so, the method500 rehydrates the instance I(S) and processes the message.

If, at step 525, a determination is made that t_(I(S))≦T, then theinstance I(S) is marked as “dehydrateable” at step 540. Dehydrateablemeans that the instance I(S) is blocked and has no executable segments,and therefore may be dehydrated without adversely affecting any activeprocesses. However, because the expected idle time t_(I(S)) does notexceed the predetermined threshold T, the resource savings that wouldresult from dehydrating the instance I(S) may not be enough to warrantcarrying out dehydration in view of the processing overhead involvedwith dehydrating the instance. As a result, a further determination asto whether the instance I(S) should be dehydrated is made in the method500 as discussed below in connection with FIG. 5B. Thus, step 540 ofFIG. 5A leads to step 555 of FIG. 5B.

As noted above, swapping out an instance I(S) reduces memoryconsumption, m, which tends to make tasks run faster on a messageprocessing system thus also reducing J and, in one embodiment, enablingthe system to operate at a higher state (e.g., A, B or C). Slowingand/or stopping activation of new tasks tends to reduce J and m. In thisway, throttling and swapping work together to reduce the strain on amessage processing system and keep tasks executing efficiently.

Referring now to FIG. 5B, the method 500 continues at step 555. In thepresent discussion of the method 500 of FIG. 5B, it will be appreciatedthat steps 555-570 require a small amount of computing power and/orsystem resources as compared to maintaining an instance in activememory, or the computing overhead associated with dehydrating andrehydrating the instance. Therefore, at step 555, the expected idle timet_(I(S)) is retrieved. It will be appreciated that between the time ofexecuting step 525 of FIG. 5A and the execution of step 560, thethreshold T may have been updated or changed. Thus, at step 560, adetermination is again made as to whether t_(I(S))>T. It will beappreciated that the time the instance I(S) has already been blocked atthe location B_(I(S)) will have increased between the time of executingstep 525 of FIG. 5A and the execution of step 560. In situations whereno data is available—for example, when no data has yet been collectedabout a particular blocking location, one embodiment of the presentinvention assumes that the expected idle time t_(I(S)) is equal to theamount of time the instance I(S) has already been blocked. In such anembodiment, therefore, the expected idle time t_(I(S)) increases as theamount of idle time increases. It will be appreciated that anyconvention may be used in situations where no data is available about ablocked location, and that such a convention may be predetermined and/ordecided upon as needed.

If the result of the determination of step 560 is the same as thedetermination of step 525 of FIG. 5A, namely that t_(I(S))≦T, then theinstance I(S) is left in active memory as at step 570, as the processingand resource benefit from dehydrating the instance I(S) is not expectedto outweigh the processing and resource overhead associated withdehydrating and rehydrating the instance I(S). If, at step 560, thedetermination is made that t_(I(S))>T, then the method 500 proceeds tostep 565. At step 565, a request to dehydrate the instance I(S) isqueued and ultimately the instance I(S) is dehydrated.

Thus, a method and system for using metrics to control throttling andswapping in a message processing system has been provided. While thepresent invention has been described in connection with the exemplaryembodiments of the various figures, it is to be understood that othersimilar embodiments may be used or modifications and additions may bemade to the described embodiment for performing the same function of thepresent invention without deviating therefrom. For example, one skilledin the art will recognize that the present invention as described in thepresent application may apply to any type or configuration of businessautomation software in any type of application environment. Therefore,the present invention should not be limited to any single embodiment,but rather should be construed in breadth and scope in accordance withthe appended claims

What is claimed:
 1. A system for improving the efficiency of a webservice message processing system, the system comprising a processor anda memory communicatively coupled to the processor, the memory comprisinginstructions that, when executed by the processor, cause the system to:determine a workload of the web service message processing system basedon performance metrics of the web service message processing system;receive a message at the web service message processing system and, inresponse to receiving the message, create an instance of a process orroute the message to an existing instance of a process; idle the createdinstance or the existing instance based on the determined workload ofthe web service message processing system; determine a predictedduration for the idling based on the performance metrics; based on thepredicted duration, move the idled instance out of active memory andinto secondary storage associated with the web service messageprocessing system; and update the determined workload based on updatedperformance metrics and-said moving the idled instance out of activememory.
 2. The system of claim 1, wherein the performance metrics arebased on memory consumed by active message processing.
 3. The system ofclaim 1, wherein the performance metrics are based on processing powerused for active message processing.
 4. The system of claim 1, whereinthe performance metrics are based on idle time of a past process.
 5. Thesystem of claim 1, wherein the performance metrics are assignedaccording to a predetermined criterion.
 6. The system of claim 1,wherein the idled instance is moved out of active memory based onwhether the predicted duration exceeds a predetermined threshold.
 7. Thesystem of claim 6, wherein the predetermined threshold is based on aprior idle process.
 8. A method for improving the efficiency of a webservice message processing system, the method comprising: determining,by the web service message processing system, a workload state based onperformance metrics of the web service message processing system; inresponse to receiving a message at the web service message processingsystem, performing one of creating an instance of a process or routingthe message to an existing instance of a process; idling the createdinstance or the existing instance based on the determined workloadstate; predicting a duration of said idling based on the performancemetrics; and moving the idled instance to secondary storage based on thepredicted duration.
 9. The method of claim 8, wherein the performancemetrics are based on memory consumed by active message processing. 10.The method of claim 8, wherein the performance metrics are based onprocessing power used for active message processing.
 11. The method ofclaim 8, wherein the storage is associated with a web service messageprocessing system.
 12. The method of claim 8, wherein the performancemetrics are based on idle time of a past process.
 13. The method ofclaim 8, wherein the performance metrics are assigned according to apredetermined criterion.
 14. The method of claim 8, wherein the systemdetermines whether the predicted duration exceeds a predeterminedthreshold.
 15. The method of claim 14, wherein the predeterminedthreshold is based on a prior idle process.
 16. A system for improvingthe efficiency of a message processing system, the system comprising aprocessor and a memory communicatively coupled to the processor, thememory comprising instructions that, when executed by the processor,cause the system to: determine a workload of the message processingsystem by accessing performance data regarding the message processingsystem, and determining, using the performance data, the workload withrespect to a system operating parameter; identify a blocked instancebeing processed by the message processing system; calculate an expectedidle time for the blocked instance by: accessing performance data forthe message processing system; determining a length of time the blockedinstance has been idle; and generating the expected idle time based onthe performance data and the length of time the blocked instance hasbeen idle; move the blocked instance to secondary storage if theexpected idle time exceeds a threshold; update the workload based onmoving the blocked instance; and update the threshold according to theupdated workload.
 17. The system of claim 16, wherein the accessedperformance data comprises at least one of: memory usage, processorpower in use by the message processing system, or assignments accordingto a predetermined criterion if no performance data is accessible. 18.The system of claim 16, further comprising instructions that, whenexecuted by the processor, cause the system to poll for a new message ata frequency according to the workload status, wherein the frequency isinversely proportional to the workload and, if the workload is above apredetermined limit, polling only for a new non-activation message. 19.The system of claim 18, wherein the polling further comprises pollingonly for a new non-activation message.
 20. The system of claim 16,wherein the blocked instance is a first instance, and the performancedata comprises a recorded idle time of a second instance.